Sparse Dictionary Learning and Domain Adaptation for Face and Action Recognition

Title of dissertation: SPARSE DICTIONARY LEARNING AND DOMAIN ADAPTATION FOR FACE AND ACTION RECOGNITION Qiang Qiu, Doctor of Philosophy, 2013 Dissertation directed by: Professor Rama Chellappa Department of Computer Science New approaches for dictionary learning and domain adaptation are proposed for face and action recognition. We first present an approach for dictionary learning of action attributes via information maximization. We unify the class distribution and appearance information into an objective function for learning a sparse dictionary of action attributes. The objective function maximizes the mutual information between what has been learned and what remains to be learned in terms of appearance information and class distribution for each dictionary atom. We propose a Gaussian Process (GP) model for sparse representation to optimize the dictionary objective function. Hence we can describe an action video by a set of compact and discriminative action attributes. More importantly, we can recognize modeled action categories in a sparse feature space, which can be generalized to unseen and unmodeled action categories. We then extend the attribute-based approach to a two-stage information-driven dictionary learning framework for general image classification tasks. The proposed method seeks a dictionary that is compact, discriminative, and generative. In the first stage, dictionary atoms are selected from an initial dictionary by maximizing the mutual information measure on dictionary compactness, discrimination and reconstruction. In the second stage, the selected dictionary atoms are updated for improved reconstructive and discriminative power using a simple gradient ascent algorithm on mutual information. When designing dictionaries, training and testing domains may often be different, due to different view points and illumination conditions. We further present a domain adaptive dictionary learning framework for the task of transforming a dictionary learned from one visual domain to the other, while maintaining a domain-invariant sparse representation of a signal. Domain dictionaries are modeled by a linear or non-linear parametric function. The dictionary function parameters and domain-invariant sparse codes are then jointly learned by solving an optimization problem. Finally, in the context of face recognition, we present a dictionary learning approach to compensate for the transformation of faces due to changes in view point, illumination, resolution, etc. The approach is to first learn a domain base dictionary, and then describe each domain shift (identity, pose, illumination) using a sparse representation over the base dictionary. The dictionary adapted to each domain is expressed as sparse linear combinations of the base dictionary. With the proposed compositional dictionary approach, a face image can be decomposed into sparse representations for a given subject, pose and illumination respectively. The extracted sparse representation for a subject is consistent across domains and enables pose and illumination insensitive face recognition. Sparse representations for pose and illumination can be used to estimate the pose and illumination condition of a face image. By composing sparse representations for subjects and domains, we can also perform pose alignment and illumination normalization. SPARSE DICTIONARY LEARNING AND DOMAIN ADAPTATION FOR FACE AND ACTION RECOGNITION